本文介绍了由波浪和太阳能运行的低成本无人面车辆（USV）的原型，该车辆可用于最小化海洋数据收集的成本。目前的原型是一个紧凑的USV，长度为1.2米，可以通过两个人部署和恢复。该设计包括电动绞盘，可用于缩回和降低水下单元。设计的几个要素利用添加剂制造和廉价的材料。通过自定义开发的Web应用，可以使用射频（RF）和卫星通信来控制车辆。通过使用先前的研究工作和先进材料的推荐，在拖曳，提升，重量和价格方面进行了优化了表面和水下装置。通过测量几个参数，例如溶解的氧，盐度，温度和pH，USV可用于水状监测。

Following the outbreak of a global pandemic, online content is filled with hate speech. Donald Trump's ''Chinese Virus'' tweet shifted the blame for the spread of the Covid-19 virus to China and the Chinese people, which triggered a new round of anti-China hate both online and offline. This research intends to examine China-related hate speech on Twitter during the two years following the burst of the pandemic (2020 and 2021). Through Twitter's API, in total 2,172,333 tweets hashtagged #china posted during the time were collected. By employing multiple state-of-the-art pretrained language models for hate speech detection, we identify a wide range of hate of various types, resulting in an automatically labeled anti-China hate speech dataset. We identify a hateful rate in #china tweets of 2.5% in 2020 and 1.9% in 2021. This is well above the average rate of online hate speech on Twitter at 0.6% identified in Gao et al., 2017. We further analyzed the longitudinal development of #china tweets and those identified as hateful in 2020 and 2021 through visualizing the daily number and hate rate over the two years. Our keyword analysis of hate speech in #china tweets reveals the most frequently mentioned terms in the hateful #china tweets, which can be used for further social science studies.

Linear classifier probes are frequently utilized to better understand how neural networks function. Researchers have approached the problem of determining unit importance in neural networks by probing their learned, internal representations. Linear classifier probes identify highly selective units as the most important for network function. Whether or not a network actually relies on high selectivity units can be tested by removing them from the network using ablation. Surprisingly, when highly selective units are ablated they only produce small performance deficits, and even then only in some cases. In spite of the absence of ablation effects for selective neurons, linear decoding methods can be effectively used to interpret network function, leaving their effectiveness a mystery. To falsify the exclusive role of selectivity in network function and resolve this contradiction, we systematically ablate groups of units in subregions of activation space. Here, we find a weak relationship between neurons identified by probes and those identified by ablation. More specifically, we find that an interaction between selectivity and the average activity of the unit better predicts ablation performance deficits for groups of units in AlexNet, VGG16, MobileNetV2, and ResNet101. Linear decoders are likely somewhat effective because they overlap with those units that are causally important for network function. Interpretability methods could be improved by focusing on causally important units.

The attention mechanism is considered the backbone of the widely-used Transformer architecture. It contextualizes the input by computing input-specific attention matrices. We find that this mechanism, while powerful and elegant, is not as important as typically thought for pretrained language models. We introduce PAPA, a new probing method that replaces the input-dependent attention matrices with constant ones -- the average attention weights over multiple inputs. We use PAPA to analyze several established pretrained Transformers on six downstream tasks. We find that without any input-dependent attention, all models achieve competitive performance -- an average relative drop of only 8% from the probing baseline. Further, little or no performance drop is observed when replacing half of the input-dependent attention matrices with constant (input-independent) ones. Interestingly, we show that better-performing models lose more from applying our method than weaker models, suggesting that the utilization of the input-dependent attention mechanism might be a factor in their success. Our results motivate research on simpler alternatives to input-dependent attention, as well as on methods for better utilization of this mechanism in the Transformer architecture.

Pre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks. Previously, for zero-shot cross-lingual evaluation, pre-trained models are only fine-tuned on English data and tested on a variety of target languages. In this paper, we do cross-lingual evaluation on various NLU tasks (sentence classification, sequence labeling, question answering) using prompt-tuning and compare it with fine-tuning. The results show that prompt tuning achieves much better cross-lingual transfer than fine-tuning across datasets, with only 0.1% to 0.3% tuned parameters. Additionally, we demonstrate through the analysis that prompt tuning can have better cross-lingual transferability of representations on downstream tasks with better aligned decision boundaries.

联合学习（FL）是标准集中学习范式的最吸引人的替代方案之一，允许异质的设备集训练机器学习模型而无需共享其原始数据。但是，FL需要中央服务器来协调学习过程，从而引入潜在的可扩展性和安全性问题。在文献中，已经提出了诸如八卦联合学习（GFL）和支持区块链的联合学习（BFL）之类的无服务器的方法来减轻这些问题。在这项工作中，我们提出了这三种技术的完整概述，该技术根据整体性能指标进行比较，包括模型准确性，时间复杂性，交流开销，收敛时间和能源消耗。广泛的模拟活动允许进行定量分析。特别是，GFL能够节省18％的训练时间，68％的能源和51％的数据相对于CFL解决方案，但无法达到CFL的准确性水平。另一方面，BFL代表了一个可行的解决方案，用于以更高级别的安全性实施分散的学习，以额外的能源使用和数据共享为代价。最后，我们确定了两个分散的联合学习实施的开放问题，并就该新研究领域的潜在扩展和可能的研究方向提供见解。

用于图像分割的深卷卷卷神经网络不会明确学习标签结构，并且可能会在类似树状结构（例如气道或血管）分割的圆柱形结构中产生不正确的结构（例如，具有断开的圆柱形结构）的分割。在本文中，我们提出了一种新型的标签改进方法，以从初始分割中纠正此类错误，并隐含地包含有关标签结构的信息。该方法具有两个新颖的部分：1）生成合成结构误差的模型，以及2）产生合成分割（带有误差）的标签外观仿真网络，其外观与实际初始分段相似。使用这些合成分割和原始图像，对标签改进网络进行了训练，以纠正错误并改善初始分割。该方法对两个分割任务进行了验证：来自胸部计算机断层扫描（CT）扫描和大脑3D CT血管造影（CTA）图像的脑血管分割的气道分割。在这两种应用中，我们的方法都大大优于标准的3D U-NET和其他先前的改进方法。当使用其他未标记的数据进行模型培训时，改进甚至更大。在消融研究中，我们证明了所提出方法的不同组成部分的值。

临床NLP任务，例如文本的心理健康评估，必须考虑社会限制 - 绩效最大化必须受保证用户数据隐私的最大重要性来限制。消费者保护法规（例如GDPR）通常通过限制数据可用性来处理隐私，例如要求将用户数据限制为给定目的的“必要内容”。在这项工作中，我们认为提供更严格的正式隐私保证，同时增加模型中用户数据量的同时，在大多数情况下，为所有涉及的各方（尤其是对用户）增加了收益。我们在Twitter和Reddit帖子的两个现有自杀风险评估数据集上演示了我们的论点。我们提出了第一个分析并置用户历史记录长度和差异隐私预算，并详细说明建模其他用户上下文如何实现公用事业保存，同时保持可接受的用户隐私保证。

学习辐射场对新型视图综合显示出了显着的结果。学习过程通常会花费大量时间，这激发了最新方法，通过没有神经网络或使用更有效的数据结构来通过学习来加快学习过程。但是，这些专门设计的方法不适用于大多数基于辐射的方法的方法。为了解决此问题，我们引入了一项一般策略，以加快几乎所有基于辐射的方法的学习过程。我们的关键想法是通过在多视图卷渲染过程中拍摄较少的射线来减少冗余，这是几乎所有基于辐射的方法的基础。我们发现，在具有巨大色彩变化的像素上的射击不仅显着减轻了训练负担，而且几乎不会影响学到的辐射场的准确性。此外，我们还根据树中每个节点的平均渲染误差将每个视图自适应地细分为Quadtree，这使我们在更复杂的区域中动态射击更多的射线，并具有较大的渲染误差。我们在广泛使用的基准下使用不同的基于辐射的方法评估我们的方法。实验结果表明，我们的方法通过更快的训练获得了与最先进的可比精度。

联邦学习（FL）引起了人们对在存储在多个用户中的数据中启用隐私的机器学习的兴趣，同时避免将数据移动到偏离设备上。但是，尽管数据永远不会留下用户的设备，但仍然无法保证隐私，因为用户培训数据的重大计算以训练有素的本地模型的形式共享。最近，这些本地模型通过不同的隐私攻击（例如模型反演攻击）构成了实质性的隐私威胁。作为一种补救措施，通过保证服务器只能学习全局聚合模型更新，而不是单个模型更新，从而开发了安全汇总（SA）作为保护佛罗里达隐私的框架。尽管SA确保没有泄漏有关单个模型更新超出汇总模型更新的其他信息，但对于SA实际上可以提供多少私密性fl，没有正式的保证；由于有关单个数据集的信息仍然可以通过在服务器上计算的汇总模型泄漏。在这项工作中，我们对使用SA的FL的正式隐私保证进行了首次分析。具体而言，我们使用共同信息（MI）作为定量度量，并在每个用户数据集的信息上可以通过汇总的模型更新泄漏有关多少信息。当使用FEDSGD聚合算法时，我们的理论界限表明，隐私泄漏量随着SA参与FL的用户数量而线性减少。为了验证我们的理论界限，我们使用MI神经估计量来凭经验评估MNIST和CIFAR10数据集的不同FL设置下的隐私泄漏。我们的实验验证了FEDSGD的理论界限，随着用户数量和本地批量的增长，隐私泄漏的减少，并且随着培训回合的数量，隐私泄漏的增加。